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CHAPTER  I 

BELIEF  IN  THE  EXISTENCE  OF 
INTERVIEWER-BIAS 

It  is  the  belief  of  many  people  who  work  in  the  field  of  public 
opinion  polls  and  market  research  surveys  that  the  interviewers  who 
are  used  in  the  studies  have  an  important  effect  on  the  results  they 
obtain.  This  effect  is  most  frequently  referred  to  as  * 'interviewer- 
bias."  Since  the  term  Mas  might  be  considered  as  indicating  an 
intentional  effect,  that  term  is  avoided  here,  and  is  replaced  by  the 
non-committal  effect. 

Whether  or  not  the  interviewer-effect  is  intentional,  its  presence 
would  be  far-reaching  and  its  detection  and  understanding  would 
be  important.  Although  there  has  been  no  unequivocal  evidence 
that  demonstrates  the  existence  of  the  effect  many  of  the  men  who 
are  concerned  with  the  administration  of  straw  ballot  procedures 
take  its  existence  for  granted,  and,  without  claiming  to  understand 
it,  endeavor  to  avoid  it. 

A  number  of  writers  have  ventured  opinions  about  interviewing 
technique  in  general  and  the  interviewer-effect  in  particular. 
Typical  of  this  type  of  article  is  one  by  Wechsler  (11),  appearing 
in  1940,  in  which  he  states  that  the  interviewer's  place  may  be  more 
important  than  any  other  aspect  of  the  public  opinion-measuring 
process.  He  points  out  that  interviewers  may  differ  in  understand- 
ing their  function  when  respondents  do  not  know  what  the  inter- 
viewer is  talking  about.  The  interviewer  may  consider  that  part 
of  his  task  is  the  explanation  of  his  question-matter,  or,  on  the 
other  hand,  that  he  is  required  simply  to  obtain  a  record  of  the 
extent  of  popular  interest  and  feeling  regardless  of  areas  of  igno- 
rance and  half-knowledge.  Wechsler  reports  that  people  seem  to 
avoid  giving  the  Don't  Know  reply  and  will  often  venture  opinions 
on  topics  far  beyond  their  understanding.  Some  interviewers, 
therefore,  may  prompt  their  respondents.  This  forcing  of  opinion 
might  be  in  the  direction  of  the  interviewer's  own  opinion.  But 
even  if  he  follows  instructions  and  reads  the  questions  verbatim, 
without  adding  a  single  word,  the  tone  of  voice  used  by  him  may 
suggest  an  answer.  This  effect,  Wechsler  suggests,  might  be  more 
striking  in  war-time,  when  feelings  on  some  issues  run  high. 

Tryon  (10)  in  1941,  in  a  paper  presented  to  a  meeting  of  the 
Western  Psychological  Association,  stressed  the  importance  of  dis- 
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covering  the  prestige  and  social  effects  of  poller  on  police,  when  both 
live  in  the  same  community  and  when  the  ballot  is  open.  Here, 
Tryon  believes,  will  be  found  the  basis  for  errors  in  the  Gallup 
election  polls. 

In  this  connection,  it  might  be  noted  that  Dr.  Gallup  himself 
raised  the  question  of  standardized  tests  for  checking  on  the  inter- 
viewer during  one  of  the  round  table  discussions  at  meetings  held 
to  celebrate  the  twentieth  anniversary  of  the  founding  of  the 
Psychological  Corporation  (12).  In  reply.  Dr.  Frank  Stanton  dis- 
cussed the  importance  of  the  problem  and  described  his  own  experi- 
ment (vide  infra).  Dr.  Henry  Link  of  the  Psychological  Corpora- 
tion explained  that  that  organization  attempts  to  surmount 
difficulties  inherent  in  interviewer-effect  by  using  many  inter- 
viewers, each  of  whom  do  rather  few  interviews. 

Blankenship  (3)  holds  that  the  problem  of  interviewing  is  "one 
of  the  greatest  questions  of  the  entire  procedure. ' '  He  regards  as 
extremely  important  "the  possible  influence  of  interviewer-bias  in 
the  results  obtained  by  the  experienced  and  trained  investigator." 

Of  the  actual  research  done  in  this  field,  almost  all  has  been  done 
in  the  laboratory  or  has  dealt  with  the  social  workers'  type  of 
interview. 

For  example,  Clark  (4)  in  1927  used  two  college  students  as 
interviewers  and  193  freshmen  as  interviewees,  the  task  being  to 
determine  the  amount  of  time  spent  in  various  activities  such  as 
athletics,  eating,  social  affairs,  etc.,  etc.  He  found  significant  dif- 
ferences between  the  two  interviewers  in  the  time  reported  as 
having  been  spent  in  different  activities.  Clark  explains  these  dif- 
ferences in  several  ways.  Apparently  the  interviewers  did  not  have 
in  mind  precisely  the  same  definitions  of  the  different  items. 
Prompting,  too,  seemed  to  play  a  part  and  suggestions  on  the  part 
of  his  interviewers  are  considered  by  Clark  as  being  "well-nigh 
unavoidable."  Such  prompting  was  most  often  done  to  stimulate 
responses  which  were  slow  in  coming.  Finally,  Clark  reports  that 
the  two  sets  of  data  were  different  "because  of  the  preconceived 
notion  of  the  interviewers."  For  example,  one  interviewer  was 
athletically  inclined;  he  found  more  time  spent  in  athletics  than 
did  the  other  interviewer.  Clark  concludes  that  his  study  demon- 
strates "the  necessity  of  taking  into  consideration  the  influence  of 
the  interviewer  on  the  data  obtained." 

What  might  be  regarded  as  the  classic  paper  in  this  field  was 
published  by  Rice  (8)  in  1929  and  consists  of  an  analysis  of  some 
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data  obtained  in  1914.  Rice  assumes  that  "contagious  bias,"  or  se- 
lective interests,  are  present  in  both  interviewer  and  respondent. 
To  investigate  this  hypothesis  he  analyzed  the  data  gathered  when 
2,000  homeless  men  were  studied  for  their  physical,  mental,  and 
social  characteristics.  These  data  had  been  collected  by  12  "skilled" 
investigators  who  had  been  assembled  from  among  the  male  staffs 
of  several  social  agencies.  Each  applicant  was  separately  inter- 
viewed by  one  of  the  investigators,  in  an  interview  which  lasted 
from  20  to  30  minutes.  The  assignment  of  applicants  to  inter- 
viewers was  random.  A  four-page  schedule  composed  of  questions 
relating  to  the  social  and  industrial  history  was  filled  out  in  the 
ease  of  each  applicant.  Some  years  later  Rice  had  occasion  to  study 
the  data  and  became  curious  about  the  presence  of  certain  uniform 
types  of  answers  in  the  records  of  men  interviewed  by  certain  of 
the  investigators.  The  questionnaire  had  provided  for  a  statement 
of  the  applicant's  own  explanation  of  his  destitution  as  well  as  for 
the  investigator's  explanation.  Rice  tabulated  the  replies  obtained 
by  two  of  the  investigators,  classifying  the  reasons  given  as  either 
"liquor"  or  "industrial"  {i.e.,  applicant's  dependency  was  due  to 
some  industrial  situation  for  which  he  was  not  individually  respon- 
sible, e.g.,  "layoff,"  "seasonal  work,"  "shut-down  of  plant,"  etc.). 
Investigator  A  reported  that  in  a  preponderance  of  cases  liquor  had 
been  the  cause  of  the  difficulties.  Furthermore,  fully  one-third  of 
the  men  he  interviewed  had  themselves  ascribed  their  destitution  to 
liquor.  Investigator  B,  on  the  other  hand,  found  that  the  primary 
cause  for  dependency  in  his  cases  could  be  traced  back  to  industrial 
factors,  and  according  to  his  report,  he  found  that  a  mere  one-tenth 
of  the  men  named  liquor  as  the  cause  of  their  downfall,  while  three- 
fifths  laid  the  blame  on  industrial  factors.  Since  the  two  inter- 
viewers had  been  questioning  the  same  type  of  man,  chosen  at 
random  from  the  group,  these  differences  struck  Rice  as  being  sur- 
prising. Upon  inquiry  he  found  that  Investigator  A  was  an  ardent 
believer  in  prohibition,  while  Investigator  B  was  regarded  by  his 
associates  as  a  Socialist. 

It  was  not  until  eleven  years  later,  in  1940,  that  another  article 
in  this  field  was  published.  Blankenship  (2)  at  this  time  presented 
a  reanalysis  of  some  material  published  earlier  the  same  year  (1), 
reporting  an  experiment  in  which  10  interviewers  obtained  300  in- 
terviews each,  on  a  questionnaire  dealing  with  elections  that  were 
about  to  be  held  in  the  area  in  which  the  opinion  poll  was  conducted. 
These  "experienced"  interviewers  had  all  answered  the  question- 
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naire  before  beginning  the  study.  In  the  reanalysis  he  compared 
the  personal  answers  of  3  interviewers  with  the  answers  they 
brought  back  for  their  respondents.  While  the  results  secured  by 
the  different  interviewers  were  found  to  be  essentially  the  same, 
there  were  7  out  of  31  eases  where  real  differences  existed  and  in 
3  of  these  cases  there  was  a  positive  correlation  between  the  inter- 
viewers' opinions  and  the  data  they  secured.  This  led  Blankenship 
to  conclude  that  "The  work  certainly  indicates  the  need  for  a  re- 
search upon  the  possible  influence  the  interviewer  may  have  on  the 
respondent's  answers."  His  own  study  was  far  from  conclusive, 
nor  was  it  meant  to  be,  since  it  was  set  up  to  investigate  an  entirely 
different  problem. 

In  1941,  Katz  (6)  conducted  a  study  intended  to  investigate  one 
aspect  of  the  problem.  His  survey  was  done  with  the  cooperation 
of  Gallup 's  American  Institute  of  Public  Opinion.  Katz  set  out  to 
discover  the  effect  of  the  social  status  of  the  interviewer  on  the 
answers  he  brought  back.  For  the  purpose  he  used  two  groups  of 
interviewers.  One  group  was  composed  of  five  of  Gallup 's  regular 
white-collar  workers,  together  with  four  similar  individuals  who 
had  applied  for  work  with  Gallup.  The  ''working-class"  staff  was 
composed  of  11  individuals,  described  by  Katz  as  "wage  workers, 
some  of  them  mill  hands,  and  most  of  them  with  no  college  train- 
ing."  These  two  groups  were  sent  into  the  same  low-rental  areas 
of  Pittsburgh  with  identical  instructions  and  questionnaires.  Katz 's 
results  indicate  differences  between  the  groups  in  the  selection  of 
respondents.  Furthermore,  there  were  differences  in  the  data 
brought  back  in  respect  to  attitudes  of  conservatism,  labor  issues, 
union  problems,  and  isolationism. 

Recently  Stanton  and  Baker  (9)  have  published  the  results  of  a 
laboratory  experiment  conducted  8  years  ago  (1934)  at  the  Ohio 
State  University.  A  learning  task  was  provided,  consisting  of  the 
presentation  on  a  screen  of  12  geometrical  forms  (nonsense  figures), 
one  at  a  time,  to  200  undergraduate  students.  Five  days  later,  and 
again  two  weeks  later  the  subjects  were  interviewed  in  the  lab- 
oratory. 

The  researchers  secured  the  services  of  three  men  and  two  women 
who  are  described  as  "experienced  interviewers,  having  had  at  least 
one  year's  experience  in  the  conduct  of  market  surveys,  and  public 
opinion  polls."  They  were  kept  in  ignorance  of  the  true  purpose 
of  the  experiment,  but  in  their  instructions  they  were  warned  to 
maintain  a  strictly  objective  and  unbiased  attitude  in  the  interview. 
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The  interviews  were  conducted  individually.  Each  interviewer 
had  before  him  a  record  sheet  giving  the  "correct"  answers.  The 
subject  was  shown  a  duplicate  of  each  of  the  original  figures  to- 
gether with  its  mirror  image.  The  interviewers  recorded  the  state- 
ment of  the  subject  as  to  which  of  each  pair  he  had  seen  in  the 
original  presentation.  In  a  carefully  balanced  arrangement,  some 
of  the  ''correct"  answers  provided  for  the  interviewers  were  incor- 
rect. The  bias  in  this  study,  then,  was  simply  the  interviewers'  pre- 
determined set  as  to  which  of  each  pair  of  figures  had  actually  been 
exposed  originally. 

Stanton  and  Baker  found  that  this  bias  on  the  part  of  the  inter- 
viewer did  have  an  effect  in  the  personal  interview  even  though  the 
interviewer  was  aware  of  the  bias  and  had  been  carefully  warned 
to  guard  against  it.  The  effect  of  the  bias  was  found  to  be  greater 
on  material  that  had  been  incompletely  learned  or  remembered. 
The  data,  however,  do  not  justify  a  conclusion  regarding  individual 
differences  in  the  effect  of  bias  among  experienced  interviewers. 

The  authors  suggest  that  minimal  cues  and  errors  in  recording 
may  be  assumed  to  account  for  most  of  the  effect. 

THE  PROBLEM  INVESTIGATED 

This  study  was  set  up  to  determine  the  existence  of  the  inter- 
viewer-effect in  public  opinion  polls  and  market  research  surveys. 
If  the  effect  were  found,  the  intention  was  to  outline  a  procedure  to 
study  and  eliminate  it. 


CHAPTEE  II 

METHOD  OF  THE  STUDY 

The  general  procedure  adopted  for  this  study  was  to  conduct  a 
well-controlled  poll  which  had  the  features  representative  of  both 
a  public  opinion  and  a  market  research  survey.  One  exception  to 
the  usual  technique  was  that  in  this  study  the  opinions  of  the  inter- 
viewers on  the  questions  they  were  to  ask  in  the  field  were  deter- 
mined in  advance.  A  second  exception  was  the  use  of  two  surveys, 
the  first  with  no  instructions  whatever  as  to  commercial  sponsorship, 
the  second,  with  definite  (false)  information  concerning  the  sponsor. 

To  fulfill  the  requirements  set  up  by  the  nature  of  the  project, 
a  questionnaire  was  especially  devised.  Since  both  public  opinion 
and  market  research  surveys  were  under  consideration,  both  types 
of  questions  were  included;  the  first  4  were  of  the  public  opinion 
type,  the  remaining  4  required  brand  names  as  answers  and  were 
of  the  market  research  type.  Question  1  was  included  as  a  "buffer" 
question,  in  accordance  with  the  usual  practice  in  this  work.  Ques- 
tion 2  was  intended  to  continue  the  trend  of  thought  established  in 
Question  1,  but  in  addition  to  deal  with  a  problem  of  current  sig- 
nificance. This  led  quite  naturally  to  a  question  (Number  3)  deal- 
ing with  another  aspect  of  war-time  America,  namely  the  sugar- 
rationing  situation  which  had  just  confronted  the  residents  of  this 
country.  A  question  concerning  rationing  suggested  immediately 
one  relating  to  metal  salvage  (Number  4). 

Following  naturally  after  such  a  question  were  the  remaining 
four  questions  dealing  with  dentifrices.  Use  of  this  product  in 
the  survey  had  two  other  advantages.  Dentifrices  are  widely  sold 
in  this  country,  and  while  people  do  have  preferences  among  the 
many  dentifrices  on  the  market,  most  individuals  are  prepared  to 
change  brands  fairly  readily.  Therefore,  it  was  felt,  an  interviewer- 
effect  might  best  be  manifested  in  connection  with  such  a  mutable 
preference. 

Question  5  was  intended  to  be  a  record  of  the  brand  names  that 
had  impressed  themselves  on  the  respondents.  In  such  a  question, 
concerned  with  partial  memory,  interviewer-effect  would  have  mini- 
mal difficulty  in  operating.  The  first  advertising  slogan  of  Question 
6  is  that  of  Ipana  toothpaste,  the  second,  of  Dr.  Lyon's  toothpowder, 
and  the  third,  of  Forhan's  toothpaste.  The  first  and  third  are  the 
brand  names  used  critically  later  in  the  study.     These  particular 
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dentifrices  were  chosen  because  of  the  fact  that  their  names  and 
slogans  have  long  been  heavily  advertised.  Furthermore,  Ipana  is 
representative  of  the  most  popular  dentifrices,  while  Forhan's  is 
one  of  the  least  widely  used  nationally  advertised  toothpastes.  The 
Dr.  Lyon's  slogan  was  used  as  a  form  of  control.  Questions  7  and 
8  were  intended  to  supplement  one  another.  If  no  interviewer-effect 
were  found  in  Question  7  (and  it  was  not  expected  that  many 
respondents  would  report  using  a  dentifrice  for  which  the  inter- 
viewer had  a  bias),  it  was  considered  that  the  bias  might  exhibit 
itself  in  the  answer  to  Question  8.  The  answer  to  Question  7,  how- 
ever, had  to  be  known  in  order  to  permit  the  interpretation  of  the 
results  obtained  for  Question  8,  in  the  event  that  an  interviewer- 
effect  was  found. 

At  the  bottom  of  the  Questionnaire  forms,  blanks  were  provided 
for  the  interviewer  to  record  the  following  facts  about  the  respon- 
dents :  socio-economic  level,  estimated  age,  sex,  marital  status,  spe- 
cific occupation.  (If  the  respondent  was  not  working  the  interviewer 
was  to  record  the  occupation  of  the  head  of  the  family.)  Finally, 
the  interviewer  was  to  record  the  place  of  the  interview,  and  sign 
his  name.  Each  questionnaire  was  numbered  by  machine  to  provide 
a  code  record  which  indicated  when  the  interview  was  obtained,  and 
by  whom.    The  questionnaire  read  as  follows : 

1.  Have  you  gone  to  the  movies  this  week?    Yes — No — 
(If  yes)     How  often? 

(If  no)     Would  you  have  gone  if  you  had  the  time?    Yes — No — DK — 

2.  Do  you  enjoy  war  stories  in  the  movies?    Yes — No — DK — 

(If  yes)     Would  you  like  to  see  more  of  them?     Yes — No — DK — 

(If  no)    Do  you  like  to  see  newsreels  dealing  with  the  war?  Yes — No — DK — 

3.  Do  you  feel  that  one-half  a  pound  of  sugar  a  week  is  sufficient  for  your 
needs?     Yes— No— DK — 

Do  you  think  that  rationing  is  a  satisfactory  method  for  controlling  sugar 
distribution?     Yes — No — DK — 

4.  Do  you  feel  that  turning  in  old  toothpaste  tubes  when  you  buy  toothpaste 
is  an  important  contribution  to  wartime  metal  salvage?    Yes — No — DK — 

5.  What  brand  of  toothpaste  or  toothpowder  or  liquid  dentifrice  have  you  seen 
or  heard  advertised  lately?    Brands 

6.  What  brand  of  dentifrice  advertises     ' '  For  the  smile  of  beauty  . . . "  ?  

What  brand  of  dentifrice  advertises     "  Do  as  your  dentist  does  . . . "  ?  

What  brand  of  dentifrice  advertises     "  ...  4  out  of  5  may  get  it ..."  ?  

7.  What  brand  of  dentifrice  do  you  use? 

8.  If  your  favorite  brand  of  dentifrice  were  all  sold  out  what  brand  would 
you  buy  instead?  

A Estimated  age  Specific  occupation 

B 
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C Man (If    respondent    is    not 

D Woman working,    get   occupation 

OE-wh Married of  head  of  family.) 

CI Single Place    

OE-cl 


Interviewer's  Name 


The  subjects  in  this  study  were  the  interviewers.  The  people 
interviewed  (respondents)  were  not  the  subjects  of  study,  nor  were 
their  opinions  of  primary  interest  here.  Therefore,  the  selection 
of  interviewers  was  of  utmost  importance. 

On  the  basis  of  the  experience  of  individuals  who  bave  long 
worked  in  this  field,  it  was  predicted  that  inexperienced  interviewers 
would  provide  results  of  doubtful  significance  in  a  study  of  this 
type.  If  the  interviewer-effect  is  observed  in  the  case  of  untrained 
interviewers,  it  can  be  argued  that  this  is  simply  a  function  of  their 
lack  of  training.  Hence  it  was  deemed  necessary  for  the  purpose 
of  this  investigation  to  use  people  who  were  typical,  experienced, 
trained,  professional  interviewers ;  these  requirements  were  fulfilled 
by  the  interviewers  whose  services  were  secured.  It  was  advisable 
to  have  them  typical  in  order  to  duplicate  actual  field  conditions 
as  closely  as  possible;  experienced,  so  that  problems  of  self-confi- 
dence, rapport  and  so  on,  would  not  arise;  trained,  so  that  each 
interviewer's  approach  to  the  mechanics  of  interviewing  would  be 
as  uniform  as  possible ;  professional,  so  that  the  job  in  hand  would 
be  taken  seriously,  and  not  merely  as  a  means  of  earning  pin-money 
with  a  minimum  of  effort. 

The  interviewers  used  were  obtained  from  a  list  provided  by  the 
National  Opinion  Kesearch  Center,  a  non-profit,  non-commercial 
organization  established  at  the  University  of  Denver.  The  field 
workers  used  by  this  organization  are  carefully  selected,  indi- 
vidually and  personally  trained,  and  constantly  refreshed  in  their 
technique,  both  by  mail  and  by  personal  visits  from  supervisors. 
Many  of  them  devote  their  full  time  to  public  opinion  surveys. 
Some  work  for  Elmo  Roper 's  Fortune  surveys,  some  for  Princeton 's 
Office  of  Public  Opinion  Research,  as  weU  as  for  the  N.  0.  R.  C. 
The  fact  that  these  people  were  accustomed  to  working  for  more 
than  one  organization  was  a  definite  advantage  in  this  study. 

All  of  the  interviewers  filled  out  the  bottom  part  of  the  ques- 
tionnaire sent  to  them,  thus  giving  complete  information  as  to  their 
own  estimations  of  their  economic  status,  and  their  age,  sex,  and 
marital  status.  This  is  shown  in  Table  I  together  with  an  indication 
of  the  sponsorship  group  into  which  each  was  placed.     It  will  be 
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TABLE  I 
Interviewers'  Descriptions  of  Themselves 


Inter- 
viewer's 
Number 


City 


Economic 
Status* 


Age 


Sex 


GroupX 


Status 


1 

Augusta,  Georgia 

B 

25 

F 

M 

I 

2 

San  Angelo,  Texas 

C 

37 

F 

S 

I 

3 

Muskogee,  Oklahoma 

B 

27 

F 

S 

I 

4 

Denver,  Colorado 

B 

35 

F 

S 

F 

5 

Dallas,  Texas 

C 

31 

F 

M 

F 

6 

Huntington,  West  Virginia 

B 

45 

F 

M 

I 

7 

Milwaukee,  Wisconsin 

C 

24 

F 

S 

I 

8 

New  Orleans,  Louisiana 

C 

30 

F 

M 

I 

9 

Denver,  Colorado 

c 

41 

F 

M 

F 

10 

Tulsa,  Oklahoma 

B 

26 

F 

M 

F 

11 

Pittsburgh,  Pennsylvania 

C 

50 

M 

M 

I 

12 

Columbia,  South  Carolina 

B 

45 

F 

M 

I 

13 

Wichita,  Kansas 

C 

33 

F 

M 

F 

14 

New  Orleans,  Louisiana 

B 

31 

F 

M 

I 

15 

Columbia,  South  Carolina 

B 

38 

F 

M 

I 

16 

Peoria,  Illinois 

C 

33 

F 

M 

F 

17 

Kansas  City,  Missouri 

B 

27 

F 

M 

F 

18 

Charlotte,  North  Carolina 

B 

38 

F 

M 

F 

19 

Beading,  Pennsylvania 

B 

35 

F 

M 

F 

20 

Tulsa,  Oklahoma 

B 

46 

F 

M 

F 

21 

Pittsburgh,  Pennsylvania 

B 

30 

F 

M 

I 

22 

Wichita,  Kansas 

C 

43 

F 

S 

F 

*  See  Appendix. 

X  I  indicates  that  the  interviewer  was  told,  immediately  before  the  second 
survey,  that  the  Ipana  toothpaste  people  were  sponsoring  the  dentifrice  ques- 
tions.    Similarly,  F  stands  for  Forhan's. 

noted  that  the  group  is  almost  evenly  distributed  in  respect  to  eco- 
nomic status,  13  reporting  themselves  in  the  B  group,  and  9  in  the 
C  group.  In  their  instructions  they  were  told  to  obtain  half  of  their 
interviews  from  respondents  in  each  of  these  economic  groups,  thus 
leaving  out  of  consideration  the  problem  raised  in  Katz's  (6)  study 
where  the  interviewers  were  of  a  different  economic  level  from  the 
respondents.  Furthermore,  all  but  one  of  the  interviewers  were 
female,  and  thus  no  question  as  to  sex  differences  between  inter- 
viewers can  be  raised.  All  but  five  were  married,  and  of  these  five 
at  least  one  may  have  been  a  widow,  divorced,  or  separated.  (The 
interviewers'  instructions  on  this  point  were  to  report  as  "single" 
any  individual,  married  or  not,  who  was  not  living  with  a  spouse.) 
Since  part  of  the  plan  of  this  study  involved  misleading  the 
interviewers  to  the  extent  of  naming  false  sponsors,  it  is  obvious 
that  no  existent  commercial  or  research  organization  could  have 
been  named.  Therefore,  some  neutral  organization  had  to  be  set 
up,  which  could,  conceivably,  engage  in  commercial  surveys.     For 
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use  in  this  work  a  business  name  and  address  were  adopted  and 
stationery  was  printed  with  the  following : 

HENEY  BLAKE  &  ASSOCIATES 
AMERICAN  SURVEYS 
15  Amsterdam  Avenuei 
New  York  City 

The  National  Opinion  Research  Center  provided  the  names  of  47 
interviewers.  A  carefully  mimeographed  copy  of  the  following 
letter,  on  letterhead,  was  sent  to  each  person  on  the  list. 

May  1,  1942 
Miss  Jane  Jones 
12  Third  Avenue 
Tulsa,  Oklahoma 

Dear  Miss  Jones: 

Through  Mr.  Harry  Field  of  the  National  Opinion  Research  Center,  of  the 
University  of  Denver,  I  have  learned  that  you  are  experienced  at  interviewing, 
and  I  would  like  to  know  if  you  can  be  available  for  a  survey  to  be  conducted 
during  the  week  of  May  11.  Inasmuch  as  you  will  have  to  do  only  30  interviews, 
taking  about  ten  minutes  each,  your  entire  job  should  be  completed  in  just  a 
few  hours. 

Although  you  were  recommended  by  Mr.  Field,  this  survey  is  in  no  way 
connected  with  the  National  Opinion  Research  Center.  The  usual  standards  of 
the  N.  O.  R.  C.  will,  however,  apply — except  where  specifically  stated  to  the  con- 
trary. For  your  cooperation  in  this  survey,  you  will  be  compensated  at  the 
usual  N.  0.  R.  C.  rates — 65^  per  hour  and  in  the  event  that  you  need  to  travel, 
the  cost  of  public  transportation,  or  five  cents  per  mile  for  your  own  car. 

If  you  feel  that  references  are  needed,  you  are  at  liberty  to  check  with  Mr. 
Field  or  Mr.  Williams  at  Denver,  or  Mr.  Sheatsley  at  the  New  York  office. 

No  hard-to-meet  deadline  will  be  set,  so  that  you  will  be  able  to  complete 
your  quota  in  your  most  convenient  time,  within,  of  course,  reasonable  limits. 

You  will  find  enclosed,  a  sample  of  the  questionnaire.  If  you  are  willing 
to  undertake  the  work,  will  you  please  fill  out  your  own  answers  to  this  ques- 
tionnaire and  mail  it  back  to  me,  as  soon  as  possible,  in  the  enclosed  stamped, 
self -addressed  envelope?  As  soon  as  I  get  this  from  you,  I  shall  send  specifica- 
tions and  a  supply  of  blank  questionnaires. 

I  have  no  doubt  that  you  will  find  it  both  pleasant  and  easy  to  work  on 
this  survey  with  us. 

A  sample  copy  of  the  questionnaire  was  included.  Of  the  47  indi- 
viduals contacted,  28  immediately  replied,  indicating  their  willing- 
ness to  do  the  work,  and  enclosing  a  copy  of  the  questionnaire  with 
their  own  answers.    To  these  28  were  mailed  envelopes  containing : 

1  Thanks  are  due  to  Dr.  Paul  F.  Lazarsfeld,  for  permission  to  use  this 
address,  that  of  the  Columbia  University  Office  of  Radio  Research,  as  a  mailing 
address. 
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1.  A  stamped,  addressed  return  envelope — (A  rubber  stamp  bearing  the  name 
of  the  ' '  organization  ' '  had  been  prepared  for  this  purpose,  in  order  to  main- 
tain the  illusion  of  a  large  business  firm.) 

2.  35  copies  of  the  questionnaire  form — (numbered  by  machine  for  positive 
identification). 

3.  A  time-and-expense  report — (This  was  modelled  after  that  used  by  the 
N.  O.  R.  C.  to  simplify  the  interviewer's  statement  of  his  charges  for  the 
work.  It  provided  for  the  name  and  address  of  the  interviewer,  a  statement 
of  the  days  actually  spent  on  the  survey,  a  breakdoAvn  of  the  hours  required, 
and  an  indication  of  transportation  charges.) 

4.  A  quota  sheet — (This  also,  was  modelled  after  that  used  currently  by  the 
N.  O.  R.  C.  The  policy  of  the  Denver  organization  in  regard  to  assignment 
of  quotas  permits  almost  complete  control  of  the  makeup  of  the  groups  of 
respondents.  The  instructions  to  interviewers  contained  on  this  sheet  were 
that  i  of  the  30  interviews  required,  should  be  obtained  from  respondents  in 
the  B  socio-economic  level  and  the  remaining  i  in  the  C  socio-economic  level.2 

In  addition  20  of  the  respondents  should  be  under  40  years  of  age,  and  10 
should  be  over  40  years  of  age.  This  particular  proportion  of  ages  was  set- 
tled upon  as  being  simplest  for  the  interviewers  to  obtain.  At  any  rate  this 
proportion  remained  constant  for  all  interviewers  throughout  the  whole  sur- 
vey. Finally,  instructions  called  for  half  of  the  respondents  to  be  male,  and 
half  female.  The  quota  sheet  as  designed,  greatly  simplified  the  interview- 
er's task  in  keeping  a  record  of  this  three-way  classification.) 

5.  A  "  specification ' '  sheet — (again  modelled  after  that  used  by  the  N.  O.  R.  C.) 
This  read  as  follows: 

HENRY  BLAKE  &  ASSOCIATES 

AMERICAN  SURVEYS 

15  Amsterdam  Avenue 

New  York  City 

SPECIFICATIONS 

You  will  find,  in  addition  to  this  specification  sheet,  a  supply  of  question- 
naire forms  (sufficient  to  complete  your  quota  in  the  study  which  you  agreed, 
earlier  this  week,  to  work  on),  a  quota  sheet,  a  time-and-expense  report,  and  a 
stamped  return  envelope. 

Since  we  have  been  assured  of  your  competence  as  an  interviewer,  we  would 
naturally  expect  that  all  of  the  interviews  assigned  to  you  be  completed  by  you. 
yourself.  We  ask  that  you  have  your  completed  forms  in  the  maU  not  later 
than  Wednesday,  May  20. 

Throughout  the  survey,  you  should,  of  course,  adhere  to  the  N.  O.  R.  C 
standards,  unless  specifically  stated  to  the  contrary.  For  example,  all  questions 
should  be  put  exactly  as  they  are  worded  on  the  questionnaire,  etc. 

THE  QUESTIONS 

Question  1.     If  answers  other  than  "yes"  or  "no"  are  given,  record  answer 
briefly. 


2  The  Interviewers  were  instructed  to  follow  the  usual  N.  O.  R.  C.  standards 
in  respect  to  socio-economic  classification.  That  organization's  instructions  on 
this  point  appear  in  full  in  the  Appendix. 
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Question  2. 


Question  3. 
Question  4. 

Question  5. 


Question  6. 


Question  7. 
Question  8. 


(If  yes)  :  If  the  respondent  answered  "yes"  to  the  first  part  of 
the  question,  then  ask  "How  often?"  and  write  figure  giving  the 
number  of  times,  e.g.,  "Twice,"  write  2. 

(If  no)  :  If  the  respondent  answered  "no"  to  the  first  part  of  the 
question,  then  ask  the  question  as  indicated  on  the  form.  DK  in 
this  question,  as  in  the  following  ones,  may  mean  either  "Don't 
Know, "  or  "  No  Opinion, "  or  "  Depends, ' '  depending  upon  which 
best  suits  the  meaning  of  the  answer.  In  other  words,  any  answer 
other  than  "yes"  or  "no"  should  be  recorded  by  a  check  mark 
in  the  space  after  DK. 

Ask  the  question  as  it  stands.     If  the  answer  is  * '  yes ' '  ask  the 
question  marked  (If  yes).     On  the  other  hand,  if  the  respondent 
answered  "  no "   (meaning  that  he  does  not  enjoy  war  stories  in 
the  movies)  then  ask  the  part  marked  (If  no). 
This  question  is  straightforward. 

If  there  are  any  qualifying  statements,  record  them  briefly  in  the 
space  following  the  answers. 

Eecord  in  the  space  provided,  the  names  of  all  dentifrices  respon- 
dents report  having  seen  advertised  in  magazines,  newspapers,  etc., 
or  having  heard  advertised  on  the  radio.  If  the  respondent  reports 
that  he  does  not  remember  any  dentifrice  advertising,  record  none 
in  the  answer  space. 

If  the  respondent  does  not  seem  to  understand  the  question  as  put, 
you  may  reword  the  question  by  asking  "What  dentifrice  uses  as 
its  advertising  slogan  .  .  .  ? "  If  more  than  one  name  is  given 
for  a  single  slogan,  record  all  of  them.  If  you  like,  you  may 
preface  each  part  of  Question  6  with,  ' '  Do  you  happen  to 
know.  ..." 

Write  in  the  name  of  the  brand,  or  brands,  used.  You  may,  if  you 
prefer,  begin  this  question,  "Would  you  mind  telling  me.  ..." 
Here  too,  write  in  the  name  of  the  brand  or  brands  mentioned. 


FACTUAL  INFORMATION 

The  economic  classifications,  as  indicated  here,  are  to  be  considered  in  the 
same  terms  as  those  set  down  by  the  N.  O.  R.  C,  for  instance,  OR-wli  means  a 
white  person  on  relief,  CI  means  a  colored  person,  etc. 

Estimate  the  age  of  the  respondent  as  closely  as  possible,  and  write  in  the 
figure  on  the  line  imder  the  phrase  ' '  Estimated  age. ' ' 

Check  the  respondent 's  sex  and  marital  status. 

Enter  specific  occupation  on  the  linos  provided ;  enter  the  place  in  which 
the  interview  was  made  according  to  community  and  state;  finally  enter  your 
name  or  initials  at  the  bottom  of  the  page. 

*♦»*»»*»»• 

If  it  is  absolutely  impossible  for  you  to  meet  the  May  20  deadline,  please 
try  to  mail  the  questionnaires  back  as  soon  after  that  date  as  you  can. 


After  a  period  of  about  two  weeks  27  interviewers  returned  their 
assignments  properly  completed,  while  the  remaining  one  was  never 
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heard  from  again.  The  interviewers,  it  might  be  added,  carried 
out  their  instructions  to  the  letter  with  one  minor  exception. 
When  the  completed  assignment  had  been  received,  the  following 
letter  was  sent  to  each  interviewer : 

(This  letter  was  individually  typed  in  order  to  encourage  the  belief  that  these 
extra  interviews  were  on  an  individual  basis,  whereas,  actually,  an  identical 
letter  was  sent  to  each  interviewer  who  had  cooi^erated  up  to  this  point.) 

May  25,  1942 
Miss  Jane  Jones 
12  Third  Avenue 
Tulsa,  Oklahoma 

Dear  Miss  Jones : 

Your  completed  questionnaires  have  been  received  and  we  are  pleased  to  say 
that  your  work  is  entirely  satisfactory. 

After  a  conference  with  representatives  of  the  XXXXX  Toothpaste  people, 
who  are  sponsoring  the  dentifrice  questions,  we  have  come  to  the  conclusion  that 
additional  interviews  are  desirable.  I,  therefore,  would  like  you  to  undertake 
an  assignment  similar  to  the  previous  one.  You  will  find  enclosed,  another  copy 
of  the  questionnaire.  If  you  are  able  to  accept  the  assignment,  please  fill  it  out 
and  return  it  in  the  stamped  self-addressed  envelope,  as  soon  as  possible. 

In  the  event  that  you  are  unable  to  accept  this  additional  assignment,  we 
shall  send  you  the  full  amount  asked  for  in  your  bill  when  you  inform  us  to  this 
effect.  If,  however,  you  are  willing  to  cooperate  with  us  further,  payment  wiU 
be  made  for  both  surveys  as  soon  as  we  receive  your  second  group  of  completed 
questionnaires  (unless  you  request  immediate  payment  for  the  work  you  have 
already  done). 

The  XXXXX  was  filled  in  with  ''Ipana"  in  the  case  of  14  of  the 
interviewers  and  "Forhan's"  in  the  case  of  the  other  13.  (The 
twenty-eighth  interviewer  was  to  be  classified  as  a  "Forhan's" 
interviewer,  but  failed  to  complete  the  first  survey.) 

The  decision  as  to  which  of  the  two  sponsors  was  to  be  named 
for  each  interviewer  was  determined  as  follows :  2  of  the  interviewers 
reported  using  Forhan's  toothpaste,  while  another  2  reported  using 
Ipana.  (The  remaining  23  interviewers  used  some  other  dentifrice.) 
In  order  to  provide  a  proper  balance,  one  of  the  Ipana  users  was  put 
in  the  Ipana-sponsored  category,  while  the  other  Ipana  user  was 
put  in  the  Forhan's  group;  similarly,  one  Forhan's  user  was  put 


3  One  interviewer,  Number  2,  obtained  only  21  interviews,  while  her  sister 
did  the  other  9.  Tor  purposes  of  this  study,  the  9  done  by  the  sister  were  dis- 
carded and  the  21  were  increased  proportionately  to  30.  That  is,  if  7  (i)  of 
her  respondents  answered  Yes  to  a  question,  this  was  recorded  as  10  (i  of  30) 
on  the  assumption  that  she  would  have  reported  essentially  the  same  proportion 
of  responses  had  she.  obtained  more  interviews. 
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in  the  Ipana-sponsored  category,  and  the  other  Forhan's  user,  in 
the  Forhan's  group. 

The  next  basis  for  grouping  was  the  community  in  which  the 
interviewers  lived.  In  9  instances  2  interviewers  lived  in  the  same 
community,  and  it  was  thought  advisable  to  place  each  pair  of 
interviewers  in  the  same  experimental  group.  This  was  done  to 
maintain  the  confidence  of  the  interviewers  in  the  integrity  of  the 
organization,  in  the  event  that  the  2  interviewers  living  in  the  same 
community  were  to  meet,  or  to  communicate  with  one  another  and 
discuss  the  survey  on  which  they  were  working. 

The  third  basis  for  grouping  was  that  of  sex,  but-  inasmuch  as 
in  the  group  of  27  there  were  but  3  men,  it  was  possible  merely  to 
arrange  to  have  2  men  in  the  Ipana  group,  and  one  in  the  Forhan's 
group. 

The  fourth  determining  factor  was  that  of  age.  For  the  Ipana 
group  the  range  was  22  to  50,  with  a  mean  of  32.08 ;  for  the  Forhan's 
group  the  range  was  25  to  46,  with  a  mean  of  32.83. 

Of  the  27  interviewers,  5  were  unable,  or  unwilling  for  various 
reasons,  to  undertake  the  second  survey,  but  the  other  22  answered 
favorably  and  sent  in  their  own  completed  questionnaires.  (By  a 
stroke  of  fortune,  exactly  half  of  this  group  fell  into  each  sponsor- 
ship group.)  When  the  favorable  replies  were  in,  a  second  assign- 
ment was  sent  to  each  interviewer.  This  was  identical  in  every 
respect  with  the  first  assignment,  save  that  the  specification  sheet 
gave  the  deadline  as  June  15. 


CHAPTEE  III 
RESULTS 

The  hypothesis,  in  terms  of  which  all  of  the  following  data  were 
analyzed,  assumes  that  interviewers  in  public  opinion  and  market 
research  surveys  tend  to  bring  in  results  which  favor  either  their 
own  personal  biases  or  the  bias  of  the  organization  by  which  they 
are  employed.  For  the  purposes  of  this  study,  an  interviewer's 
personal  bias  is  considered  to  be  simply  in  the  direction  of  his  own 
answer  to  a  particular  item;  thus,  an  interviewer  who  answers  a 
question  with  ' '  Yes ' '  is  called  a  *  *  Yes  interviewer. ' '  If  the  hypothe- 
sis is  true  "Yes  interviewers"  report  more  "Yes"  answers  than  do 
"No  interviewers."  This  treatment  was  applicable  to  Questions  1, 
2,  3,  4,  and  6.  In  the  last-named,  however,  since  it  dealt  with  a 
matter  of  fact,  rather  than  of  opinion,  the  interviewer's  bias  was 
considered  to  be  in  the  direction  of  the  correctness  or  incorrectness 
of  his  response. 

An  interviewer  who  answered  "Ipana"  to  the  first  slogan  of 
Question  6  was  considered  a  "Bight  interviewer."  A  response  of 
either  "Don't  Know"  or  some  other  dentifrice  name,  caused  the 
interviewer  to  be  classified  as  either  a  "Don't  Know — "  or  a 
"Wrong  interviewer,"  respectively.  Similar  treatment  was  ac- 
corded, independently,  to  each  of  the  other  two  parts  of  the  ques- 
tion, with  "Dr.  Lyon's"  being  the  correct  answer  for  the  second 
part  of  the  question  and  "Forhan's"  being  the  correct  response  for 
the  third  part. 

The  bias  described  above  may  be  called  "interviewer's  personal 
bias"  and  has  been  assumed  to  manifest  itself  in  public  opinion 
polls. 

In  market  research  surveys  the  bias  is  assumed  to  change  from 
one  in  which  the  interviewer  brings  back  responses  agreeing  with 
his  own  opinion,  to  one  in  which  the  interviewer  brings  back  results 
that  tend  to  place  his  sponsor  in  a  favorable  light.  Such  a  bias 
may  be  called  "interviewer's  sponsorship  bias."  (Both  of  these 
types  are  subsumed  in  "interviewer-effect"  insofar  as  this  study 
is  concerned.) 

The  questions  falling  under  this  latter  type  of  bias  are  5,  6,  7, 
and  8.  In  this  study  two  possible  sponsorship  biases  were  intro- 
duced:  Ipana  Toothpaste  (contained  in  the  instructions  to  11  of 
the  interviewers)  and  Forhan's  Toothpaste  (contained  in  the  in- 
structions for  the  remaining  11  interviewers.) 

21 
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(Question  6,  since  it  has  the  features  of  both  types  of  survey, 
was  treated  in  both  ways.  For  the  analysis  presently  under  con- 
sideration, the  answers  to  this  question  were  considered  not  in  terms 
of  "Right"  or  "Wrong,"  but  rather  in  terms  of  the  sponsorship 
instructions  given  the  interviewers.) 

If  this  kind  of  interviewer-effect  does  play  a  part  in  such  a 
survey,  it  would  be  expected  that  an  interviewer  with  knowledge 
of  his  sponsor  would  report  more  responses  favoring  that  sponsor 
than  he  would  without  knowledge  of  sponsorship.  In  these  terms, 
the  "Ipana"  responses  brought  in  on  the  first  survey  by  the  11 
interviewers,  who,  unknown  to  themselves  were  later  to  be  Ipana- 
instructed,  were  compared  with  the  "Ipana"  responses  brought  in 
on  the  second  survey  by  the  same  interviewers,  who  for  that  survey 
had  such  Ipana-instructions.  A  similar  procedure  was  adopted  for 
the  Forhan's  interviewers,  and  followed  throughout  the  last  four 
questions.  (In  Question  6,  the  only  parts  susceptible  to  this  anal- 
ysis were  the  first  and  third.) 

The  results  of  the  tabulation  of  the  data  indicate  that  there  was 
consistency  throughout  in  respect  to  the  proportions  of  responses 
obtained  in  the  two  surveys  for  a  given  question.  In  no  case  was 
the  difference  between  corresponding  items  greater  than  5.5%,  with 
an  average  difference  of  about  2%.  This  overall  picture  of  the 
two  surveys,  separately  and  combined,  is  presented  in  Table  II. 

Since  the  interviewers'  own  answers  to  all  questions  must  be 
known  in  detail,  a  complete  tabulation  of  each  interviewer's  re- 
sponses to  all  "public  opinion"  type  questions  on  both  surveys  was 
necessary.  Such  analysis  shows  that  for  the  most  part  the  inter- 
viewers were  consistent  in  their  responses.  The  greatest  incon- 
sistency was  found  in  the  answers  to  the  three  parts  of  Question  6 
in  which,  either  as  a  result  of  knowledge  of  the  majority  opinion 
("climbing  on  the  band-wagon"*)  or  because  of  additional  infor- 
mation as  to  the  correct  answer  (which  may  also  be  considered  a 
"band-wagon  effect"),  the  responses  were  changed.  Of  a  total  of 
27  changes  in  the  answers  to  Question  6,  19  were  made  from  either  a 
Don't  Know,  or  a  Wrong,  to  a  Right  response.  The  information 
concerning  the  interviewers'  answers  to  Questions  1,  2,  3,  4,  and  6 
can  be  found  in  Table  III.  (Questions  5,  7,  and  8  were  designed  to 
permit  study  of  the  effect  of  sponsorship-information,  discussed 
later.) 


*  This  phrase  is  used  in  the  public  opinion  field  to  refer  to  situations  where 
individuals  adopt  as  their  own,  the  majority  opinion  on  a  question. 
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TABLE  II 

Eespondent's  Answers  to  Questions  1,  2,  3,  4,  and  6 

Question  "S®        So'^'^SofOoSo 
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l_rirst  Survey  226  34.24  434  65.76 

l_Second   Survey   262  39.70  398  60.30 

Combined  Surveys  488  36.97  832  63.03 


2— First  Survey  329  49.85  299  45.30  32  4.85 

2— Second   Survey   357  54.09  268  40.61  35  5.30 

Combined  Surveys  686  51.97  567  42.95  67  5.08 

3a— First  Survey  518  78.48  117  17.73  25  3.79 

3a— Second  Survey 545  82.58  103  15.61  12  1.82 

Combined  Surveys   1063  80.53  220  16.67  37  2.80 

3b — First  Survey  561  85.00  45  6.82  54  8.18 

3b— Second  Survey 560  84.85  45  6.82  55  8.33 

Combined  Surveys  1121  84.92  90  6.82  109  8.26 

4— First  Survey  525  79.55  73  11.06  62  9.39 

4— Second   Survey   527  79.85  50  7.58  83  12.57 

Combined  Surveys  1052  79.70  123  9.32  145  10.98 
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6a— First  Survey  328  49.70  236  35.76  96  14.54 

6a— Second  Survey 330  50.00  238  36.06  92  13.94 

Combined  Surveys  658  49.85  474  35.91  188  14.24 

6b— First  Survey  237  35.91  342  51.82  81  12.27 

6b— Second  Survey 221  33.48  333  50.45  106  16.06 

Combined  Surveys  458  34.70  675  51.14  187  14.17 

6c— First  Survey  36  5.45  485  73.48  139  21.06 

6c— Second  Survey  38  5.76  478  72.42  144  21.82 

Combined  Surveys  74  5.61  963  72.95  283  21.44 


The  next  step  in  the  analysis  is  the  breaking-  down  of  the  respon- 
dents' answers  according  to  the  response  of  the  interviewer,  for 
each  question  individually.  This  is  necessary  to  study  the  relation- 
ship between  interviewers'  and  respondents'  answers.  In  the  first 
survey  almost  without  exception  the  proportions  of  answers  in  any 
given  category  were  constant  regardless  of  the  interviewers'  re- 
sponse. Thus,  an  interviewer  who  answered  "Yes"  to  a  question 
got  the  same  proportion  of  "Yes"  and  "No"  responses  as  an  inter- 
viewer who  answered  "No"  to  the  same  question. 

In  only  two  cases,  Questions  6b  and  6c,  do  the  differences  appear 
to  be  striking,  but  these  differences  can  be  attributed  to  the  extreme 
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disparity  of  the  **n's"  involved,  13  compared  to  1,  in  one  ease, 
and  9  to  1  in  the  other. 

In  the  second  survey,  however,  we  find  again  two  differences, 
Questions  1  and  2,  that  are  large.  But  in  Question  1  a  more  careful 
scrutiny  of  the  original  data  reveals  that  the  major  contribution 
to  the  mean  of  one  of  the  members  of  the  comparison  was  made  by 
two  individual  interviewers  whose  reported  ''Yes"  responses  were 
greatly  in  excess  of  the  average  of  the  group.    In  the  other  case  we 


find  an  extreme  disparity  between  **n's,"  viz.,  20 
data  can  be  found  in  Tables  IV  and  V. 

and  2. 

These 

TABLE  IV 

Eesponses  Broken  Down  According  to  Interviewee's  Eesponses 

ON  THE  First  Survey 

Inter- 
viewers' 
Besponse 

Number 

of 
Inter- 
viewers 

Number  of  Bespondents  Answering 

Question 

Number  Percent  Number 

Percent  Number 

Percent 

Yes 

No 

DK 

1. 

Yes 

No 

5 

17 

51 

175 

34.00 
34.31 

99 

335 

66.00 
65.69 

2. 

Yea 

No 

DK 

12 
9 
1 

189 

129 

11 

52.50 
47.78 
36.67 

154 

126 

19 

42.78 
46.67 
63.33 

17 

15 

0 

4.72 
5.55 
0.00 

3a. 

Yes 

No 

DK 

19 
3 
0 

448 
70 

78.60 
77.78 

101 
16 

17.72 
17.78 

21 
4 

3.68 
4.44 

3b. 

Yes 

No 

DK 

21 
1 
0 

538 
23 

85.40 
76.67 

40 
5 

6.35 
16.67 

52 
2 

8.25 
6.67 

4. 

Yes 

No 
DK 

17 
1 

4 

404 
25 
96 

79.22 
83.33 
80.00 

56 

2 

15 

10.98 

6.67 

12.50 

50 
3 
9 

9.80 

10.00 

7.50 

Bight 

Wrong 

DK 

6a. 

Right 

Wrong 

DK 

15 
1 
6 

240 
20 
68 

53.33 
66.67 
37.78 

60 

2 

34 

13.33 

6.67 
18.89 

150 

8 

78 

33.33 
26.67 
43.33 

6b. 

Right 

Wrong 

DK 

13 
1 
8 

172 

8 

57 

44.10 
26.67 
23.75 

34 

4 

43 

8.72 
13.33 
17.92 

184 

18 

140 

47.18 
60.00 
58.33 

6e. 

Right 

Wrong 

DK 

1 

9 

12 

5 
13 
18 

16.67 

4.82 
5.00 

3 

64 

72 

10.00 
23.70 
20.00 

22 
193 
270 

73.33 
71.48 
75.00 

In  the  sponsorship-bias  phase  of  the  study  a  somewhat  different 
problem  is  encountered.  Kather  than  being  interested  in  the  effect 
of  an  interviewer's  opinion  on  the  results  returned,  the  interest 
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TABLE  V 

Eesponses  Broken  Down  According  to  Interviewer's 

ON  the  Second  Survey 

Eesponses 

Inter- 
viewers ' 
Response 

Number 

of 
Inter- 
viewers 

Number  of  Respondents  Answering 

Question 

Number  Percent  Number 

Percent  Number 

Percent 

Yes 

No 

DK 

1. 

Yes 
No 

7 
15 

106 
156 

50.48 
34.67 

104 
294 

49.52 
65.33 

2. 

Yes 

No 

DK 

8 

13 

1 

139 

207 

11 

57.92 
53.08 
36.67 

91 

165 

12 

37.92 
42.31 

40.00 

10 

18 

■    7 

4.16 

4.61 

23.33 

3a. 

Yes 

No 

DK 

20 
2 
0 

502 
43 

83.67 
71.67 

86 
17 

14.33 

28.33 

12 
0 

2.00 
0.00 

3b. 

Yes 

No 

DK 

21 
1 
0 

537 
23 

85.24 
76.67 

42 
3 

6.67 
10.00 

51 

4 

8.09 
13.33 

4. 

Yes 

No 

DK 

19 

2 
1 

464 
45 
18 

81.40 
75.00 
60.00 

35 
6 
9 

6.14 
10.00 
30.00 

71 
9 
3 

12.46 
15.00 
10.00 

Bight 

Wrong 

DK 

6a. 

Eight 

Wrong 

DK 

21 

1 
0 

318 
12 

50.48 
40.00 

85 

7 

13.49 
23.33 

227 
11 

36.03 
36.67 

6b. 

Eight 

Wrong 

DK 

19 

1 
2 

195 
11 
15 

34.21 
36.67 
25.00 

82 

7 

17 

14.39 
23.33 
28.33 

293 
12 
28 

51.40 
40.00 
46.67 

6c. 

Eight 

Wrong 

DK 

7 

5 

10 

21 

5 
12 

10.00 
3.33 

4.00 

35 
44 
65 

16.67 
29.33 
21.67 

154 
101 
223 

73.33 
67.33 
74.33 

becomes  centered  on  the  relative  effects  of  varying  sponsorship 
knowledge  on  the  reported  responses  for  the  second  survey.  This 
can  best  be  determined  by  a  comparison  of  the  responses  "Ipana" 
(or  "Forhan's")  brought  back  by  "Ipana-"  (or  "Forhan's-") 
interviewers  on  the  two  surveys.  For  example,  in  Question  5  the 
eleven  interviewers  who,  on  the  second  survey  were  told  that  the 
Ipana  people  were  sponsoring  the  dentifrice  questions  reported  124 
Ipana  responses  in  the  first  survey,  when  they  had  no  sponsorship 
instructions,  as  compared  with  120  Ipana  mentions  on  the  second 
survey.  This  difference  is  obviously  not  significant.  Similarly,  in 
the  case  of  Forhan's,  we  find  that  there  are  7  Forhan's  mentions 
in  the  first  survey,  as  compared  with  12  in  the  second  survey.  A 
similar  procedure  was  followed  for  Questions  7  and  8  with  no  com- 
parison more  striking  than  the  ones  indicated  above.    This  inf  orma- 
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TABLE  VI 

Number  of  Times  Each  Product  Was  Mentioned  in  Reply  to  Questions 

5,  7,  8 — According  to  Sponsorship  Instructions  to 

Interviewers  on  the  Second  Survey 


First  Survey 


Second  Survey 


Dentifrice 


Ipana  Forhan's  Ipana  Forlian's 

Interviewers     Interviewers  Interviewers     Interviewers 


Question  5 

Ipana   

Forhan  's    ... 
Dr.  Lyons  .. 
Pepsodent 
Colgate's   ... 

Teal 

Calox    

Listerine 

Squibb 's    ... 

Phillip's  

Kolynos   

Others  

None  

D.K 

Question  7 

Ipana   

Forhan 's  ... 
Dr.  Lyons  .. 
Pepsodent ... 
Colgate's   ... 

Phillip's  

Teel 

Listerine 

Kolynos   

Squibb 's  

Dr.  West's 

Calox 

Others  

None  

D.K 

Question  8 

Ipana   

Forhan 's 

Dr.  Lyons  .. 
Pepsodent 
Colgate's   ... 

Teel 

Calox 

Listerine 

Squibb 's  

Phillip's  

Dr.  West's 

Pebeco    

Others  

None  

D.K 


124 
3 

66 
143 

56 

18 

2 

4 

5 

12 

5 

10 

45 

19 


73 
5 

28 

56 

42 

10 

9 

7 

2 

8 

0 

11 

41 

27 

9 


61 
2 

29 

48 

37 

11 

10 

5 

6 

5 

1 

5 

25 

33 

43 


90 

7 

33 

173 

63 

31 

9 

9 

7 

2 

10 

20 

79 

9 


37 
6 

23 

69 

38 

5 

8 

7 

5 

18 

7 

22 

42 

42 

6 

44 
2 

28 

48 

40 

16 

4 

11 

7 

7 

6 

3 

10 

40 

62 


120 

2 

48 

149 

60 

21 

7 

17 

5 

7 

6 

15 

64 

4 

85 
2 

41 

48 

44 

8 

6 

10 

1 

5 

1 

20 

32 

24 

15 

55 
3 

29 

53 

43 

3 

5 

16 

7 

2 

6 

4 

14 

26 

69 


83 

12 

45 

145 

61 

26 

7 

13 

6 

4 

2 

26 

68 

11 


38 

8 

31 
63 
45 

4 
20 
11 

4 
11 

9 
18 
44 
44 
19 


35 
6 

21 

67 

41 

5 

8 

4 

7 

7 

8 

2 

20 

33 

62 
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tion,  together  with  a  complete  tabulation  of  the  dentifrices  named 
in  this  study  are  included  in  Table  VI. 

There  a  separate  tabulation  is  presented  for  all  dentifrices  which 
are  mentioned  at  least  5  times  on  each  survey,  or  failing  that,  at  least 
ten  times  in  one  of  the  surveys.  The  "others"  category  includes 
all  dental  products  which  were  mentioned  so  infrequently  as  to  fail 
in  satisfying  these  requirements.  The  **none"  category  includes 
actual  responses  of  "no  dentifrice,"  as  well  as  "salt,"  "salt  and 
soda,"  "soap,"  etc.,  in  addition  to  specialized  products  made  for 
false  teeth.  Since  the  nature  of  these  questions  was  such  that  the 
respondent  was  required  to  answer  with  one  (or  more)  brand 
name(s),  the  total  number  of  responses  indicated  for  any  given 
question  may  be  different  from  the  total  number  of  respondents. 

For  the  verification  of  the  hypothesis  it  was  necessary  to  obtain 
estimates  of  the  significance  of  the  differences  in  three  different 
situations.    They  were : 

1.  Differences  between  "Yes"  responses  of  "Yes-interview- 
ers" and  "Yes"  responses  of  "No-interviewers." 

2.  Differences  between  "Right"  responses  of  "Right-interview- 
ers" and  "Right"  responses  of  "Wrong-interviewers." 

3.  Differences  between  "Ipana"  responses  brought  in  by 
"Ipana-interviewers"  on  the  first  survey  and  "Ipana" 
responses  brought  in  by  "Ipana-interviewers"  on  the  second 
survey. 

Differences  between  "Forhan's"  responses  brought  in  by 
"Forhan's-interviewers"  on  the  first  survey  and  "Forhan's" 
responses  brought  in  by  "Forhan's-interviewers"  on  the 
second  survey. 

The  "t"  test  was  used  for  the  determination.^  The  results  of  the 
application  of  this  test  are  given  in  Table  VII.  In  all  cases  but  2 
the  differences  are  not  significant,  virtually  all  of  the  "t's"  failing 
to  meet  even  the  10%  level. 

In  one  case.  Question  3a  on  the  second  survey,  the  difference  is 
apparently  significant  at  the  3%  level.  Little  faith,  however,  can 
be  had  in  the  "t"  test  in  this  instance,  since,  as  was  pointed  out 
above,  one  of  the  groups  whose  means  were  compared  consisted  of 
only  2  interviewers.  In  the  other  case.  Question  1  on  the  second 
survey,  the  difference  is  significant  at  about  the  1%  level.  Notwith- 
standing this  "t"  value,  a  question  may  again  be  raised  as  to  its 

5  Cf .  Fisher,  E.  A. :  Statistical  Methods  for  Eesearch  Workers.  Edinburgh 
and  London:  Oliver  and  Boyd,  1936.     Chapter  V. 
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interpretation  as  a  statistic,  bearing  in  mind  the  fact  that  the  size 
of  the  mean  is  largely  a  result  of  the  responses  obtained  by  only  two 
interviewers  whose  reported  "Yes"  responses  were  greatly  in  excess 
of  the  average  reported  by  all  the  "Yes-interviewers." 

These  isolated  instances  do  not  vitiate  the  conclusion  that  no 
interviewer-effect  is  apparent  in  this  study. 

Although  no  interviewer-effect  was  found  upon  individual 
treatment  of  the  questions,  there  was  the  possibility  that  some  over- 
all effect  might  be  found  in  which  those  interviewers  whose  opinions 
were  those  of  the  majority,  brought  back  more  majority  responses 
than  did  those  interviewers  whose  opinions  coincided  with  the 
minority.  "With  this  in  mind,  the  majority  responses  reported  by 
interviewers  who  held  majority  opinions  were  compared  with  the 
majority  responses  reported  by  interviewers  who  themselves  held 
minority  opinions,  after  lumping  all  of  the  public  opinion  type  ques- 
tions. This  treatment  did  not  indicate  any  significant  differences 
and  in  no  way  altered  conclusions  otherwise  derived. 


CHAPTEE  IV 

SUMMARY  AND  CONCLUSIONS 

Twenty-two  interviewers  obtained  660^  interviews  on  each  of  two 
surveys.  The  first  survey  was  completed  without  information  as  to 
the  sponsor.  The  second  survey  was  made  after  the  interviewers 
had  been  supplied  with  (false)  information  concerning  the  sponsor- 
ship of  certain  of  the  questions.  Before  each  survey  was  started, 
the  interviewers  reported  their  own  opinions  and  answers  to  each 
question. 

It  was  found  that  neither  opinions  on  questions  nor  knowledge 
of  sponsorship  affected  the  results  in  this  survey.  The  differences 
in  the  results  brought  back  by  interviewers  of  different  opinions, 
and  between  interviewers  with  different  knowledge  of  sponsorship 
were  not,  on  the  whole,  significant.  The  test  of  significance  used, 
"  t, "  indicated  that  the  differences  that  were  found  could  very  well 
be  accounted  for  on  a  chance  basis  and  therefore  they  cannot  be 
said  to  support  the  hypothesis  of  interviewer-effect. 

It  would  seem,  then,  that  the  conclusion  reached  here  is  counter 
to  the  common  belief  that  interviewers  almost  invariably  affect  their 
data  in  ways  favorable  to  the  opinions  they  themselves  hold,  or  feel 
they  ought  to  hold  in  deference  to  the  individuals  or  organizations 
sponsoring  the  study.  This  belief  is  so  widespread  that  many  pub- 
lic opinion  and  market  research  directors  take  steps  to  minimize  the 
opportunity  for  the  effect  to  occur. 

The  only  previously  published  field  study  in  favor  of  the  belief 
is  the  study  by  Katz  (6)  in  which  it  was  shown  that  the  social  status 
of  the  interviewer  was  associated  with  the  results  brought  back.  In 
analyzing  his  results  it  appears  that  much  of  the  difference  could 
be  ascribed  to  selection  of  respondents  (a  problem  that  did  not  enter 
the  present  study).  Some  of  the  differences  in  Katz's  work  can  be 
traced  back  to  different  attitudes  associated  with  the  different  social 
classes.  Again,  this  factor  was  controlled  in  this  study  in  order  to 
investigate  interviewer-effect  in  a  broader  sense.  It  is  apparent 
that  in  this  study  where  the  interviewers  were  of  the  same  socio- 
economic class  as  the  respondents  (in  contrast  with  Katz's),  the 
effect  was  not  found.  To  do  away  with  interviewer-effect,  therefore, 
one  of  the  requirements  would  seem  to  be  that  of  using  interviewers 
who  are  homogeneous  with  respect  to  the  individuals  they  are  to 
interview. 


6  Please  see  footnote  on  page  19. 
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Other  evidence  favoring  the  belief  in  the  existence  of  the  inter- 
viewer-effect comes  from  studies  such  as  those  done  by  Stanton  and 
Baker  (9)  and  the  one  analyzed  by  Rice  (8).  In  these  cases  a  few 
"trained"  interviewers  discussed  at  some  length  certain  problems 
or  experiences  with  individuals  seated  at  the  other  side  of  a  table. 
Under  these  conditions  the  interviewer-effect  was  found.  But  this 
conclusion,  limited  to  a  laboratory  study,  does  not  conflict  with  the 
present  research,  done  as  it  was  by  a  greater  number  of  interviewers 
going  from  door  to  door  and  asking  brief  questions  which  could  be 
answered  with  a  single  word,  or  little  more.  The  present  study 
bears  more  resemblance  to  public  opinion  polls  and  market  research 
surveys  than  do  the  laboratory  interview  studies,  and  indeed,  the 
investigators  in  the  latter  studies  do  not  attempt  to  generalize  their 
findings  to  include  the  door-to-door  survey. 

In  this  study  the  interviewers  were  thoroughly  trained  in  the 
best-known  techniques  of  interviewing.  Before  being  placed  on  the 
staff  of  the  organization  which  had  provided  their  names,  the  inter- 
viewers were  required  to  read  a  comprehensive  manual  describing 
the  public  opinion  process  and  the  best  methods  of  handling  the 
exacting  task  of  interviewing.  In  addition,  the  interviewers  were 
individually  trained  by  supervisors,  and  were  constantly  being  re- 
freshed and  brought  up  to  date  by  pamphlets  which  were  mailed  to 
them  regularly.  There  is  general  agreement  that  the  interviewers 
used  are  among  the  best  in  the  country. 

A  limitation  of  this  study  is  the  probable  fact  that  there  was  but 
little  personal  involvement  of  the  interviewers  in  the  questions.  (It 
is  unfortunate  that  in  some  questions  there  was  an  unequal  division 
of  opinion  among  the  interviewers,  resulting  in  a  comparison  of  a 
large  group  with  a  small  group.)  While  it  is  true  that  the  inter- 
viewers had  opinions  on  the  various  questions,  and  that  they  knew 
who  the  "sponsor"  was,  there  is  no  evidence  to  show  that  they  re- 
garded the  questions  as  important,  or  that  the  sponsor  was  of  great 
significance  to  them.  It  remains  for  further  studies  to  show 
whether  questions  with  greater  personal  involvement  will  result  in 
an  interviewer-effect.  It  will  probably  take  much  experimentation 
to  find  where  the  dividing  line  lies  between  involvement-producing- 
interviewer-effect  and  less-involvement-resulting-in-no-interviewer- 
effect. 

Similarly  it  remains  for  other  studies  to  determine  the  kind  of 
training  and  the  minimum  amount  of  training  needed  to  result  in  a 
poll  free  from  interviewer-effect.     A  subject  for  further  investiga- 
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tion  is  consideration  of  the  possibility  of  an  interviewer-effect  that 
brings  about  results  which  are  counter  to  the  opinions  of  the  inter- 
viewers. And,  of  course  left  entirely  to  other  studies,  is  the  ques- 
tion of  selection-bias.  The  conclusion  from  this  study  is  considered 
to  be  limited  to  the  demonstration  that,  contrary  to  popular  belief, 
it  is  possible  to  conduct  a  poll  or  survey  in  which  the  interviewer- 
effect  is  not  an  important  uncontrolled  variable. 
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APPENDIX 

The  National  Opinion  Research  Center  issues  basic  instructions 
for  interviewers  which  deal  with  almost  all  phases  of  the  interview- 
ing process.  The  instructions  regarding  economic  classification 
stress  the  fact  that  an  individual's  standard  of  living  is  not  neces- 
sarily determined  by  his  own  income  or  earning  power  alone,  but 
rather  by  the  total  income  of  the  family  of  which  he  is  a  member, 
by  the  size  of  the  family,  and  by  the  community  in  which  he  lives. 
Therefore,  the  N.  0.  R.  C.  does  not  use  income  level  alone  to  desig- 
nate the  standard  of  living,  but  instead  uses  economic  groups. 
These  groups  are  defined  as  follows : 

The  A  Group  is  composed  of  the  wealthy  and  prosperous  fam- 
ilies in  the  community.  These  people  have  all  the  comforts  and 
necessities  that  money  can  buy,  and  most  or  all  of  the  luxuries  com- 
mon to  the  community.  (The  word  "luxuries"  should  not  be  inter- 
preted in  too  rare  a  sense.  They  are  not  restricted  in  our  definition 
to  a  yacht,  or  a  stable  of  horses,  or  a  hunting  preserve,  etc.  "We 
mean  families  able  to  afford  some  of  the  following  things  (or  similar 
things)  :  a  servant  or  servants,  membership  in  the  country  club,  a 
second  house  at  the  shore,  on  the  lake,  or  in  the  mountains,  a  trip 
to  Florida  or  Canada  in  season,  etc.)  The  A's  run  from  million- 
aires down  to  anyone  with  sufficient  income  to  afford  any  reasonable 
luxury  sometimes  enjoyed  in  his  part  of  the  country.  Depending 
on  the  community,  these  constitute  from  4  to  8%  of  the  population. 

Class  B  is  the  next  lower  group.  Like  the  A's,  these  families 
have  all  the  comforts  and  necessities.  They  differ  from  the  A's  va 
that  they  have  to  pick  and  choose  between  the  luxuries.  In  other 
words,  if  a  luxury  runs  into  considerable  money,  they  have  to  weigh 
one  luxury  against  another.  The  B  group  typically  constitutes 
from  20  to  25%  of  the  community. 

The  C's  are  the  great  middle  class.  These  people  have  all  of  the 
necessities  of  life  and  most  of  the  comforts,  but  typically  they  are 
not  able  to  afford  luxuries.  Some  C's  occasionallj^  save  enough  to 
reach  up  for  a  few  simple  luxuries.  The  C's  represent  from  45  to 
50%  of  the  population. 

The  D  Oroup  is  composed  of  poor  people.  Often  they  have  ir- 
regular employment,  or  if  regular,  their  wages  are  at  the  bottom  of 
the  scale.  They  cannot  afford  many  of  the  comforts,  or  even  neces- 
sities, of  life.  On  many  surveys  part  of  your  D  quota  will  be  com- 
posed of  those  "On  Relief."     This  means  people  on  home  relief, 
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W.  P.  A.  projects  (not  supervisory  jobs,  however),  and  those  receiv- 
ing old  age  benefits.  The  D  's  then,  scale  from  those  who  can  obtain 
some  of  the  necessities  of  life  with  a  struggle  on  down  to  '  *  reliefers. ' ' 
D's  constitute  20  to  25%  of  the  population. 

The  CI  Group  represents  colored  people  (negroes).      Only  clas- 
sify them  as  such.     Do  not  designate  them  economically. 
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